Applicability of MAHOUT for Large Data Sets

In an attempt to unify two hot trends in the research community, this project explores distributed data mining. The idea of taking big data and processing it to extract information on a large number of nodes has many applications for many sectors outside of computer science. This project explores Apache Mahout and attempts to quantify its modifiability, accuracy, and performance. To determine the modifiability, the code is opened up and examined. To get an idea of the accuracy, the resulting output is scored and compared against the highest-scoring algorithm in an online competition for the same dataset. Finally, the performance is examined to see how well Mahout scales. This project shows the importance of understanding the fundamentals of any data mining solution before attempting to use it. Poor parameters and a general lack of understanding led to interesting results and, more importantly, a number of useful ”lessons learned”.